Evaluation of Regressive Methods for Automated Generation of Test Trajectories

Automated generation of test cases is a prerequisite for fast testing. Whereas the research has addressed the creation of individual test points, test trajectoiy generation has attracted limited. In simple terms, a test trajectoiy is defined as a series of data points, with each (possibly multidimensional) point relying upon the value(s) of previous point(s). Software systems that use data trajectories as inputs include closed-loop process controllers. For these systems, software testers can either handcraft test trajectories, use input trajectories from older versions of the system or, perhaps, collect test data in a high fidelity system simulator. While these are valid approaches, they are expensive and time-consuming, especially if the assessment goals require substantial number of tests. In this paper, we propose a framework for expanding a small, conventionally developed set of test trajectories into a large set suitable, for example, for system safe0 assurance. In the core of this framework is statistical regression analysis. The regression analysis builds a relationship between controllable independent variables and closely correlated dependent variables, which represent test trajectories. By perturbing the independent variables, new test trajectories can be generated automatically. Automated test trajectory generation has been applied in the safety assessment of a fault tolerant flight control system. We compare the performance of simple linear regression, multiple linear regression, and autoregressive techniques. The peiformance meti-ics include the speed of test generation and the percentage of “acceptable” trajectories, measured by the domain specific reasonableness checks.